HomeBlogsAncelus Database Blog

High Performance Blog

Updates on the world of high performance organizations and the information they use.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Team Blogs
    Team Blogs Find your favorite team blogs here.
  • Login

Posted by on in Database

Recently had a question on the market goal of TQL.  They wanted to know: Are you trying to replace SQL?

TQL should be viewed as an extension to the basic SQL function set.  But there are some data/query situations where existing technology isn't up to the task. The goal of TQL is to solve these problems by allowing Ancelus performance to come to bear without disturbing the existing code base.

TQL does not replicate all the functions of SQL, but adds capability to handle complex threaded views at very high speed.  

In most cases a customer has called us because of a few functions in the existing SQL application that have failed to scale due to complexity or size.  In those cases we add the Ancelus/TQL toolset to address those specific problems only, with no modification to the majority of the SQL code.  The intercept of data and query is usually non-intrusive, and it simply appears to be really fast.  

The ability of Ancelus/TQL to convert the ERD view into a defined table view with high performance makes it ideal for this task.

Hits: 25

Posted by on in Database

The TQL toolset is headed back to beta after some modifications.  The link below offers a brief demo.  Comments welcome.

The driving reason for the development of TQL was the ability of the Ancelus database to handle far more complex structures than any other database.  Traditional query methods simply wouldn't work.  The secondary reason was the desire to preserve the exceptional speed of Ancelus in the developed query.  It's finally come together.

Note: Video inoperable in Firefox.  Use Chrome, IE or Edge.

TQL Short Demo


Hits: 111

Posted by on in Database

The HDF5 format has become a de facto standard for handling scientific data.  While it has all the advantages of standardized methods, plus a recently added streaming capability, it also has some serious limitations.

Hierarchical Data Format has to date been built on the assumption of post collection analysis. Ancelus now introduces Dynamic HDF5 capability that greatly expands functionality.

1. Stream data in and query data at the same time.

2. In-stream statistical analysis at up to 1 million statistical ops per second.  Supports a two layer strategy with instant anomaly detection leading to more efficiency in the complex analytics.

3. Stream indexing completely eliminates downtime to re-index critical data tables.

Hits: 436

Posted by on in Database

There are many critical applications that require a dual commit capability with assured completion.  This involves processing a transaction to two computers at remote locations with ACID level assurance of completion at both sites.

Even though no standard has been finalized for this function, the tentative measure shows Ancelus at 200 nanoseconds plus network latency.  

This timing excludes the TCP/IP stack (or it's replacement). Since there are multiple network protocols with widely divergent performance, it is unlikely that we will see an end-to-end standard soon.


Hits: 454

Posted by on in Database

Ancelus Transaction Processing tools have completed another round of testing.  Many strong modifications came from the beta site feedback.  Ancelus now includes protections against all (we think) forms of deadlock. It also includes messaging to help troubleshoot some of the common problems in application development.

Of particular importance is the performance.  Performance loss was only 1% with real-time ACID compliance on the persistent store.

It's now back for a second round of beta evaluation.  Integration into the TQL user interface is the next priority.

Hits: 382

Posted by on in Database

Testing of the transaction processing (TXP) functions is nearing completion.  This set of utilities supports multi-lock for relational writes, and retains ACID compliance through real-time journal management.  With these additions Ancelus may be the easiest system to use for TXP application development.

The most important aspect of TXP is that there is no performance penalty because of the unique locking processes used by Ancelus.  Relational views can now operate at speeds previously restricted to streaming applications. 

A new part of the specification for Ancelus is the ability to start or stop the journal file in-stream.  It wasn't originally envisioned to operate this way, but in very large datasets it makes a major contribution to simpler administration and less downtime.

The final step in the TXP delivery will be to integrate it into TQL.


Hits: 1326

Posted by on in Database

Yesterday we reviewed the completed Phase 1 release of the Java integration modules.  Ancelus now supports both JNI and JDBC integration.  

Both of these integrations are tightly coupled with the TQL (threaded query language) development tool unique to Ancelus.  TQL was specifically developed to retain the extreme performance of the Ancelus database.  It also has the advantage of reducing query development time from days to minutes  The point-and-click screens deliver a fully optimized query and an instant results sample. It prevents most common sources of error and ambiguity inherent in SQL and other relational-centric query tools.  For example, it eliminates any need for developer knowledge of database internals or table structure.

We're ready to go to beta testing with the Java tools, but haven't resolved a site.  Most Ancelus customers develop applications in php or C, including our own RT3 and A3 applications.  

If you would like to participate in this testing program, give us a call. 

Hits: 1410
Posted by on in Database

Beta testing of the TQL editor is now in progress at a live site.  Stay tuned.

Hits: 781

Posted by on in Database

A cautionary note.  AncelusDB is not supported on CentOS 6.7.  This OS has a bug that corrupts the memory map and is guaranteed to cause the application to fail unpredictably.  

Users of 6.7 should immediately upgrade to CentOS 6.8 or higher.

Hits: 1289

Posted by on in Database

The barriers have finally been broken down.  Primary architecture is defined and prototype is functional.  We had gotten distracted by an attempt to keep the query language completely independent of the physical data store. This is the foundation assumption of SQL.  Once we let go of that it didn't take long.

Since more than 90% of queries do not require that degree of independence, a new data store concept can get us out of the trap of being held hostage to the needs of the 10%.  And no performance penalties.  The 10% of queries that are really "ad-hoc" (not defined by the application) will be slower than the 90% cases.  But still faster than the all systems based on the relational assumption.

The packaging and clean up to get this to beta level will take several months, but it now looks like we're on the right track.  

Hits: 1176

Posted by on in Database

Our development work on TQL continues but not without some setbacks.  Several strategies have been abandoned in the past 18 months, and the idea of keeping it an arms-length toolset (independent of the database) is now on the shelf.  

TQL development will now move down a new path, but with less disclosure.  We think we're on a track that will result in a new operating model.

Watch this space.  We'll let you know as soon as we can.

Hits: 1179

Posted by on in Database

A common objection heard from IT organizations when discussing Ancelus is "Our database and technology stack is fast enough."  

So how fast is "fast enough"?  This usually means the user response time doesn't generate too many complaints.  But that's a red herring.

When it comes to database performance Speed = $$$.  Accepting a relational system as "fast enough" usually means "lots of hardware solved it."  We had one customer that reduced server count from 252 to 26.

The more precise question would seem to be "How much technology bloat would you like to fund?"


Hits: 1180

Posted by on in Database

The new benchmarks are now published on the benchmarks page.

Described as "amazing" and "impressive" by industry analysts.

Hits: 1146

Posted by on in Database

What would happen if we tested the world's fastest database on the world's fastest server?

We usually ignore the announcements of the hardware and chip manufacturers.  A debate about a 3 GHz clock vs. a 3.1 GHz clock has no significant effect on Ancelus performance. Our unique architecture eliminates most of the operating system and CPU functions.

That's why we chose to publish our benchmarks on a 1U pizza box, mail order server that cost under $8,000. 

But something happened last week that is changing our attitude.  We received a new server based on the Intel Broadwell CPU. This is the unquestioned leader in server performance.  We're now in the process of running new benchmarks. We're suddenly paying a lot of attention to this hardware.  

First results suggest that Intel has done something very fundamental.  Slower clock speed (2.2 GHz) but much higher performance.

Hits: 1681

Posted by on in Database

The NoSQL movement has received a lot of attention over the past few years.  But the reality is turning out to be different from the hype.

All of these systems start with an assumption of a relational model.  Most are table based, a few are columnar/relational.  They all fall into the same trap.  They assume the issue is with SQL and that they need to find a faster way to do SQL.  They haven't yet gotten to the real problem.

The relational assumption is the core problem.  It doesn't match the native state of information in nature, so it requires translation routines.  The problem isn't that SQL is badly designed.  SQL is forced to do massive transforms because of the relational model.  Trying to make SQL faster misses the point that SQL does massive amounts of unproductive work.  In normalized relational structures (many tables) the amount of unproductive work expands exponentially.  In de-normalized relational structures (one or few tables) the schema structure is an add on.  In either case scaled performance decays exponentially.

Ancelus eliminates all these problems by eliminating the relational storage structure and replacing it with a mathematical model.  The physical storage model is purely abstract.  The logical structure is purely native information - linked and recursive lists. Columns and tables are abstractions, mathematically derived.

The result is that instead of handling data many times, discarding the uninteresting, Ancelus only handles the interesting stuff - the final results set.  Dramatic reduction in computing and network load.

Hits: 2192

Posted by on in Database

Ancelus 6 is released.  Includes performance improvements.  New benchmarks will publish soon.

Hits: 1592

Posted by on in Database

The latest update for the Ancelus database, version 6.0, has been installed in beta mode at a customer site.  Includes major improvements in varchar speed and memory utilization.

GA release target date is June 1. 


Hits: 1727

Posted by on in Database

There seem to be two trends in dealing with the explosion of data from the Internet of Things.  The dominant theme seems to be focused on the efficient administrative tasks in handling bigger datasets.  This approach has no end game that we can see, since the most successful examples are de-normalized data structures struggling to deal with the write-speed constraints.  This means you've duplicated massive amounts of data.  So you solve the excess data problem by increasing the amount of data?  

Hadoop and Cassandra use two different approaches, but they both involve a vast array of hardware.  And neither has a reputation for addressing the real issue of cutting "time-to-insight."

The second is the still-small use of streaming analytics. Do it on the fly and focus on the efficiency of the data scientist rather than the admin. Most of these solutions deliver limited scope, but we've taken a different approach.  First we use the extreme performance of Ancelus to enable more precise flagging of interesting trends.  First statistics operate on the live data stream.  Then we support executable stored procedures to accelerate the data scientist's in-depth analysis, but only after pointing to the area of interest.  No fishing expeditions needed.

Our streaming toolset is called A3: Ancelus Adaptive Analytics and it serves both purposes without the hardware explosion implied by physically de-normalized data.  The Ancelus logical structure is 100% normalized and 100% de-normalized at the same time.  No data duplication, no pre-defined storage structure, so the logical structure is unconstrained.  It can even be changed on the fly without downtime.

New technology, new game.

Hits: 3739

Posted by on in Database

The white paper at the link explains the root cause of the massive overload emerging from the Internet of Things, how it will affect current practice, and what to do about it.



Hits: 2232

Posted by on in Database

The Ancelus database has always been available to run on hosted servers, Amazon Web Services being the most common.  But we've launched a project this week to modify the architecture to support a new pricing model.

Our first step is to implement our demo system on AWS to eliminate the need for a local install.  Local systems often ran into version conflicts between php and Apache that took time to sort.  The new system will be pre-installed on Red Hat and will be immediately operable.  Sign up and start using it. By using AWS we can implement large memory, disk and CPU configurations for temporary use without having to own the hardware. 

The second step will be to add tracking functions to support transaction pricing and other usage based pricing strategies. This seems to be the other major complaint against site installed systems. 

This will also let us offer specific benchmark data comparing various cloud offerings. We continue to hear of major performance differences, but with no current way to validate how it affects Ancelus.

Stay tuned.  This could be fun.

Hits: 3341
Go to top View Our Stats