Make the number of internal keeper states selectable by an optional parameter passed to require.

6 files changed, 71 insertions, 57 deletions

diff --git a/CHANGES b/CHANGES
index 1850415..bc0368a 100644
--- a/CHANGES
+++ b/CHANGES
@@ -3,9 +3,12 @@ CHANGES:
 
 CHANGE X:
 
+CHANGE 29 BGe 21-Feb-2011
+    Make the number of internal keeper states selectable by an optional parameter passed to require.
+
 CHANGE 28 BGe 18-Feb-2011
-    - moved keeper-related code in a separate source file
-    - keeper.lua is now embedded in text form instead of bytecode to improve LuaJIT2-compatibility
+    Moved keeper-related code in a separate source file
+    keeper.lua is now embedded in text form instead of bytecode to improve LuaJIT2-compatibility
 
 CHANGE 27 BGe 17-Feb-2011
     - we know Lanes is loaded in the master state, so we don't force it
@@ -26,7 +29,7 @@ CHANGE 26 BGe 14-Feb-2011:
     inter-state data copy for unsupported types
 
 CHANGE 25 BGe 12-Feb-2011:
-    Changed idfunc signature and contract to clarify that fact it is not lua-callable
+    Changed idfunc signature and contract to clarify the fact it is not lua-callable
     and to be able to require the module it was exported from in the target lanes
 
 CHANGE 24 DPtr 25-Jan-2011:
@@ -56,7 +59,7 @@ CHANGE 19 BGe 2-Dec-2010:
 
 CHANGE 18 BGe 6-Oct-2010:
     Fixed 'memory leak' in some situations where a free running lane is collected before application shutdown
-        A bit of code cleanup
+    A bit of code cleanup
 
 CHANGE 17 BGe 21-Sept-2010:
     Fixed stupid compilation errors.
diff --git a/docs/index.html b/docs/index.html
index a28dd41..3d2ecf2 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -56,7 +56,7 @@
 
 <p><br/><font size="-1"><i>Copyright &copy; 2007-11 Asko Kauppi. All rights reserved.</i>
     <br>Lua Lanes is published under the same <A HREF="http://en.wikipedia.org/wiki/MIT_License">MIT license</A> as Lua 5.1.
-    </p><p>This document was revised on 12-Feb-11, and applies to version 2.0.11.
+    </p><p>This document was revised on 21-Feb-11, and applies to version 2.1.0.
 </font></p>
 
 </center>
@@ -817,12 +817,12 @@ can be used for custom userdata as well. Here's what to do.
       <tt>"delete"</tt>: receives this same pointer on the stack, and should cleanup the object.</li>
     <li>
       <tt>"metatable"</tt>: should build a metatable for the object. Don't cache the metatable
-yourself, Lanes takes care of it ("metatable" should only be invoked once).</li>
+      yourself, Lanes takes care of it ("metatable" should only be invoked once).</li>
       <li>
         <tt>"module"</tt>: is the name of the module that exports the idfunc,
-to be pushed on the stack as a string. It is necessary so that Lanes can require it in
-any Lane and keeper state that receives a userdata. This is to prevent crashes in situations
-where the module could be unloaded while the idfunc pointer is still held.</li>
+        to be pushed on the stack as a string. It is necessary so that Lanes can require it in
+        any Lane and keeper state that receives a userdata. This is to prevent crashes in situations
+        where the module could be unloaded while the idfunc pointer is still held.</li>
 </ul>
 Take a look at <tt>linda_id</tt> in <tt>lanes.c</tt>.
     </li>
@@ -917,10 +917,10 @@ Here are some things one should consider, if best performance is vital:
     merged into one main timer state (see <tt>timer.lua</tt>); no OS side
     timers are utilized.
     </li>
-    <li>Lindas are hashed to a fixed number of "keeper states", which are a locking entity.
-    If you are using a lot of Linda objects,
-    it may be useful to try having more of these keeper states. By default,
-    only one is used (see <tt>KEEPER_STATES_N</tt>), but this is an implementation detail.
+    <li>Lindas are hashed to a number of "keeper states", which are a locking entity.
+    If you are using a lot of Linda objects, it may be useful to try having more of
+    these keeper states. By default, only one is used but this is an implementation detail.
+    It is possible to <tt>require( "lanes", N)</tt> to use more keeper states.
     </li>
 </ul>
 </p>
@@ -952,6 +952,29 @@ its actual value.
 <h2 id="changes">Change log</h2>
 
 <p>
+Feb-2011 (2.1.0)
+<ul>
+  <li>Added an auto-require mechanism to ensure any deep userdata transiting in a lane causes its parent module to be required in that lane
+  <ul>
+    <li>Changed idfunc signature and contract to clarify the fact it is not lua-callable and to be able to require the module it was exported from in the target lanes</li>
+    <li>When a deep userdata idfunc requests a module to be required, manually check that it is not loaded before requiring it instead of relying on the require function's loop detection feature</li>
+    <li>When a module must be required, raise an error if the 'require' function is not found in the target state</li>
+    <li>We know Lanes is loaded in the master state, so we don't force it to be required in every lane too when a linda deep userdata is copied</li>
+  </ul>
+  </li>
+  <li>Fixed application hang-up because keeper state was not released in case of errors thrown by inter-state data copy for unsupported types</li>
+  <li>Refactor lane proxy implementation: it is now a full userdata instead of a table, and its methods are implemented in C instead of Lua
+    <ul>
+      <li>its metatable is no longer accessible</li>
+      <li>writing to the proxy raises an error</li>
+      <li>it is no longer possible to overwrite its join() and cancel() methods</li>
+    </ul>
+  </li>
+  <li>Moved keeper-related code in a separate source file</li>
+  <li>keeper.lua is now embedded in text form instead of bytecode to improve LuaJIT2-compatibility</li>
+  <li>Make the number of internal keeper states selctable by an optional parameter passed to require.</li>
+</ul>
+
 Feb-2011 (2.0.11):
 <ul>
   <li>Fixed bug where reference to Linda object was dropped for a short time (crashing if GC was run during that time).</li>
diff --git a/src/keeper.c b/src/keeper.c
index f19beed..f89c638 100644
--- a/src/keeper.c
+++ b/src/keeper.c
@@ -53,12 +53,6 @@
 /*---=== Keeper states ===---
 */
 
-/* The selected number is not optimal; needs to be tested. Even using just
-* one keeper state may be good enough (depends on the number of Lindas used
-* in the applications).
-*/
-#define KEEPER_STATES_N 1   // 6
-
 /*
 * Pool of keeper states
 *
@@ -66,12 +60,8 @@
 * bigger the pool, the less chances of unnecessary waits. Lindas map to the
 * keepers randomly, by a hash.
 */
-static struct s_Keeper GKeepers[KEEPER_STATES_N];
-
-/* We could use an empty table in 'keeper.lua' as the sentinel, but maybe
-* checking for a lightuserdata is faster.
-*/
-static bool_t nil_sentinel = 0;
+static struct s_Keeper *GKeepers = NULL;
+static int GNbKeepers = 0;
 
 /*
 * Lua code for the keeper states (baked in)
@@ -88,10 +78,13 @@ static char const keeper_chunk[]=
 *       unclosed, because it does not really matter. In production code, this
 *       function never fails.
 */
-const char *init_keepers(void)
+char const *init_keepers( int const _nbKeepers)
 {
-        unsigned int i;
-        for( i=0; i<KEEPER_STATES_N; i++ )
+        int i;
+        assert( _nbKeepers >= 1);
+        GNbKeepers = _nbKeepers;
+        GKeepers = malloc( _nbKeepers * sizeof( struct s_Keeper));
+        for( i = 0; i < _nbKeepers; ++ i)
         {
 
                 // Initialize Keeper states with bare minimum of libs (those required
@@ -104,8 +97,11 @@ const char *init_keepers(void)
                 luaG_openlibs( L, "io,table,package" );     // 'io' for debugging messages, package because we need to require modules exporting idfuncs
                 serialize_require( L);
 
-                lua_pushlightuserdata( L, &nil_sentinel );
-                lua_setglobal( L, "nil_sentinel" );
+                /* We could use an empty table in 'keeper.lua' as the sentinel, but maybe
+                * checking for a lightuserdata is faster. (any unique value will do -> take the address of some global of ours)
+                */
+                lua_pushlightuserdata( L, &GNbKeepers);
+                lua_setglobal( L, "nil_sentinel");
 
                 // Read in the preloaded chunk (and run it)
                 //
@@ -131,12 +127,12 @@ const char *init_keepers(void)
 struct s_Keeper *keeper_acquire( const void *ptr)
 {
         /*
-        * Any hashing will do that maps pointers to 0..KEEPER_STATES_N-1 
+        * Any hashing will do that maps pointers to 0..GNbKeepers-1 
         * consistently.
         *
         * Pointers are often aligned by 8 or so - ignore the low order bits
         */
-        unsigned int i= ((unsigned long)(ptr) >> 3) % KEEPER_STATES_N;
+        unsigned int i= ((unsigned long)(ptr) >> 3) % GNbKeepers;
         struct s_Keeper *K= &GKeepers[i];
 
         MUTEX_LOCK( &K->lock_);
@@ -190,10 +186,13 @@ int keeper_call( lua_State *K, char const *func_name, lua_State *L, void *linda,
 void close_keepers(void)
 {
         int i;
-        for(i=0;i<KEEPER_STATES_N;i++)
+        for( i = 0; i < GNbKeepers; ++ i)
         {
                 lua_close( GKeepers[i].L);
                 GKeepers[i].L = 0;
                 //assert( GKeepers[i].count == 0);
         }
+        if( GKeepers) free( GKeepers);
+        GKeepers = NULL;
+        GNbKeepers = 0;
 }
diff --git a/src/keeper.h b/src/keeper.h
index 66d8d7e..e959c7c 100644
--- a/src/keeper.h
+++ b/src/keeper.h
@@ -8,7 +8,7 @@ struct s_Keeper
         //int count;
 };
 
-const char *init_keepers(void);
+const char *init_keepers( int const _nbKeepers);
 struct s_Keeper *keeper_acquire( const void *ptr);
 void keeper_release( struct s_Keeper *K);
 int keeper_call( lua_State *K, char const *func_name, lua_State *L, void *linda, uint_t starting_index);
diff --git a/src/lanes.c b/src/lanes.c
index a8aba71..8b62532 100644
--- a/src/lanes.c
+++ b/src/lanes.c
@@ -4,20 +4,7 @@
  * Multithreading in Lua.
  * 
  * History:
- *      3-Jan-11  (2.0.10): linda_send bugfix, was waiting on the wrong signal
- *      3-Dec-10  (2.0.9): Added support to generate a lane from a string
- *      2-Dec-10  (2.0.8): Fix LuaJIT2 incompatibility (no 'tostring' hijack anymore)
- *      ????????  (2.0.7): Fixed 'memory leak' in some situations where a free running
- *                  lane is collected before application shutdown
- *      24-Aug-10 (2.0.6): Mem fixes, argument checking (lua_toLinda result), thread name
- *      24-Jun-09 (2.0.4): Made friendly to already multithreaded host apps.
- *      20-Oct-08 (2.0.2): Added closing of free-running threads, but it does
- *                  not seem to eliminate the occasional segfaults at process
- *                  exit.
- *          ...
- *      24-Jun-08 .. 14-Aug-08 AKa: Major revise, Lanes 2008 version (2.0 rc1)
- *          ...
- *      18-Sep-06 AKa: Started the module.
+ *      See CHANGES
  *
  * Platforms (tested internally):
  *      OS X (10.5.7 PowerPC/Intel)
@@ -64,7 +51,7 @@
  *      ...
  */
 
-const char *VERSION= "2.0.11";
+const char *VERSION= "2.1.0";
 
 /*
 ===============================================================================
@@ -1913,7 +1900,7 @@ static const struct luaL_reg lanes_functions [] = {
 /*
 * One-time initializations
 */
-static void init_once_LOCKED( lua_State *L, volatile DEEP_PRELUDE ** timer_deep_ref )
+static void init_once_LOCKED( lua_State *L, volatile DEEP_PRELUDE ** timer_deep_ref, int const nbKeepers)
 {
     const char *err;
 
@@ -1964,7 +1951,7 @@ static void init_once_LOCKED( lua_State *L, volatile DEEP_PRELUDE ** timer_deep_
         }
   #endif
 #endif
-    err= init_keepers();
+    err= init_keepers( nbKeepers);
     if (err)
     {
             luaL_error( L, "Unable to initialize: %s", err );
@@ -2001,8 +1988,11 @@ int
 #if (defined PLATFORM_WIN32) || (defined PLATFORM_POCKETPC)
 __declspec(dllexport)
 #endif
-        luaopen_lanes( lua_State *L ) {
-
+luaopen_lanes( lua_State *L )
+{
+    static volatile int /*bool*/ go_ahead; // = 0
+    int const nbKeepers = luaL_optint( L, 2, 1);
+    luaL_argcheck( L, nbKeepers > 0, 2, "Number of keeper states must be > 0");
     /*
     * Making one-time initializations.
     *
@@ -2010,7 +2000,6 @@ __declspec(dllexport)
     * there is no problem. But if the host is multithreaded, we need to lock around the
     * initializations. 
     */
-    static volatile int /*bool*/ go_ahead; // = 0
 #ifdef PLATFORM_WIN32
     {
         // TBD: Someone please replace this with reliable Win32 API code. Problem is,
@@ -2022,7 +2011,7 @@ __declspec(dllexport)
         static volatile unsigned my_number;   // = 0
         
         if (my_number++ == 0) {     // almost atomic
-            init_once_LOCKED(L, &timer_deep);
+            init_once_LOCKED(L, &timer_deep, nbKeepers);
             go_ahead= 1;    // let others pass
         } else {
             while( !go_ahead ) { Sleep(1); }    // changes threads
@@ -2036,7 +2025,7 @@ __declspec(dllexport)
             // Recheck now that we're within the lock
             //
             if (!go_ahead) {
-                init_once_LOCKED(L, &timer_deep);
+                init_once_LOCKED(L, &timer_deep, nbKeepers);
                 go_ahead= 1;
             }
         }
diff --git a/src/lanes.lua b/src/lanes.lua
index 95bdeeb..78582f9 100644
--- a/src/lanes.lua
+++ b/src/lanes.lua
@@ -52,7 +52,6 @@ local _version= assert(mm._version)
 
 local now_secs= assert( mm.now_secs )
 local wakeup_conv= assert( mm.wakeup_conv )
-local timer_gateway= assert( mm.timer_gateway )
 
 local max_prio= assert( mm.max_prio )
 
@@ -239,6 +238,7 @@ linda = mm.linda
 
 ---=== Timers ===---
 
+local timer_gateway= assert( mm.timer_gateway )
 --
 -- On first 'require "lanes"', a timer lane is spawned that will maintain
 -- timer tables and sleep in between the timer events. All interaction with