Shuffler revisited (2)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Shuffler revisited (2)

Stephen R. van den Berg
In an attempt to prepare the Shuffler for use on the database connection
in the pgsql module, I delved back into the Shuffler code again.
This time I achieved 99.9% comprehension of the code.

The result is a considerable rewrite of the inner loop.
The rewrite was necessary because:
a. The shuffle-source-management still had memory leaks.
b. The size of the buffer writes was arbitrarily large
   (depending on the input block size).
c. The way wrap_callback results were stored resulted
   in (only) mostly-correct data.
d. The data of sources were used ever so briefly
   *after* getting freed.
--
Stephen.