From 62efc4ddcbdcc8c9f4f29b7b5ae76aa66f0b96e2 Mon Sep 17 00:00:00 2001 From: cvs2svn Date: Mon, 1 Oct 2001 21:58:55 +0000 Subject: This commit was manufactured by cvs2git to create branch 'OPENBSD_3_0'. --- src/lib/libcrypto/engine/README | 278 ---------------------------------------- 1 file changed, 278 deletions(-) delete mode 100644 src/lib/libcrypto/engine/README (limited to 'src/lib/libcrypto/engine/README') diff --git a/src/lib/libcrypto/engine/README b/src/lib/libcrypto/engine/README deleted file mode 100644 index 96595e6f35..0000000000 --- a/src/lib/libcrypto/engine/README +++ /dev/null @@ -1,278 +0,0 @@ -NOTES, THOUGHTS, and EVERYTHING -------------------------------- - -(1) Concurrency and locking ... I made a change to the ENGINE_free code - because I spotted a potential hold-up in proceedings (doing too - much inside a lock including calling a callback), there may be - other bits like this. What do the speed/optimisation freaks think - of this aspect of the code and design? There's lots of locking for - manipulation functions and I need that to keep things nice and - solid, but this manipulation is mostly (de)initialisation, I would - think that most run-time locking is purely in the ENGINE_init and - ENGINE_finish calls that might be made when getting handles for - RSA (and friends') structures. These would be mostly reference - count operations as the functional references should always be 1 - or greater at run-time to prevent init/deinit thrashing. - -(2) nCipher support, via the HWCryptoHook API, is now in the code. - Apparently this hasn't been tested too much yet, but it looks - good. :-) Atalla support has been added too, but shares a lot in - common with Ben's original hooks in bn_exp.c (although it has been - ENGINE-ified, and error handling wrapped around it) and it's also - had some low-volume testing, so it should be usable. - -(3) Of more concern, we need to work out (a) how to put together usable - RAND_METHODs for units that just have one "get n or less random - bytes" function, (b) we also need to determine how to hook the code - in crypto/rand/ to use the ENGINE defaults in a way similar to what - has been done in crypto/rsa/, crypto/dsa/, etc. - -(4) ENGINE should really grow to encompass more than 3 public key - algorithms and randomness gathering. The structure/data level of - the engine code is hidden from code outside the crypto/engine/ - directory so change shouldn't be too viral. More important though - is how things should evolve ... this needs thought and discussion. - - ------------------------------------==*==----------------------------------- - -More notes 2000-08-01 ---------------------- - -Geoff Thorpe, who designed the engine part, wrote a pretty good description -of the thoughts he had when he built it, good enough to include verbatim here -(with his permission) -- Richard Levitte - - -Date: Tue, 1 Aug 2000 16:54:08 +0100 (BST) -From: Geoff Thorpe -Subject: Re: The thoughts to merge BRANCH_engine into the main trunk are - emerging - -Hi there, - -I'm going to try and do some justice to this, but I'm a little short on -time and the there is an endless amount that could be discussed on this -subject. sigh ... please bear with me :-) - -> The changes in BRANCH_engine dig deep into the core of OpenSSL, for example -> into the RSA and RAND routines, adding a level of indirection which is needed -> to keep the abstraction, as far as I understand. It would be a good thing if -> those who do play with those things took a look at the changes that have been -> done in the branch and say out loud how much (or hopefully little) we've made -> fools of ourselves. - -The point here is that the code that has emerged in the BRANCH_engine -branch was based on some initial requirements of mine that I went in and -addressed, and Richard has picked up the ball and run with it too. It -would be really useful to get some review of the approach we've taken, but -first I think I need to describe as best I can the reasons behind what has -been done so far, in particular what issues we have tried to address when -doing this, and what issues we have intentionally (or necessarily) tried -to avoid. - -methods, engines, and evps --------------------------- - -There has been some dicussion, particularly with Steve, about where this -ENGINE stuff might fit into the conceptual picture as/when we start to -abstract algorithms a little bit to make the library more extensible. In -particular, it would desirable to have algorithms (symmetric, hash, pkc, -etc) abstracted in some way that allows them to be just objects sitting in -a list (or database) ... it'll just happen that the "DSA" object doesn't -support encryption whereas the "RSA" object does. This requires a lot of -consideration to begin to know how to tackle it; in particular how -encapsulated should these things be? If the objects also understand their -own ASN1 encodings and what-not, then it would for example be possible to -add support for elliptic-curve DSA in as a new algorithm and automatically -have ECC-DSA certificates supported in SSL applications. Possible, but not -easy. :-) - -Whatever, it seems that the way to go (if I've grok'd Steve's comments on -this in the past) is to amalgamate these things in EVP as is already done -(I think) for ciphers or hashes (Steve, please correct/elaborate). I -certainly think something should be done in this direction because right -now we have different source directories, types, functions, and methods -for each algorithm - even when conceptually they are very much different -feathers of the same bird. (This is certainly all true for the public-key -stuff, and may be partially true for the other parts.) - -ENGINE was *not* conceived as a way of solving this, far from it. Nor was -it conceived as a way of replacing the various "***_METHOD"s. It was -conceived as an abstraction of a sort of "virtual crypto device". If we -lived in a world where "EVP_ALGO"s (or something like them) encapsulated -particular algorithms like RSA,DSA,MD5,RC4,etc, and "***_METHOD"s -encapsulated interfaces to algorithms (eg. some algo's might support a -PKC_METHOD, a HASH_METHOD, or a CIPHER_METHOD, who knows?), then I would -think that ENGINE would encapsulate an implementation of arbitrarily many -of those algorithms - perhaps as alternatives to existing algorithms -and/or perhaps as new previously unimplemented algorithms. An ENGINE could -be used to contain an alternative software implementation, a wrapper for a -hardware acceleration and/or key-management unit, a comms-wrapper for -distributing cryptographic operations to remote machines, or any other -"devices" your imagination can dream up. - -However, what has been done in the ENGINE branch so far is nothing more -than starting to get our toes wet. I had a couple of self-imposed -requirements when putting the initial abstraction together, and I may have -already posed these in one form or another on the list, but briefly; - - (i) only bother with public key algorithms for now, and maybe RAND too - (motivated by the need to get hardware support going and the fact - this was a comparitively easy subset to address to begin with). - - (ii) don't change (if at all possible) the existing crypto code, ie. the - implementations, the way the ***_METHODs work, etc. - - (iii) ensure that if no function from the ENGINE code is ever called then - things work the way they always did, and there is no memory - allocation (otherwise the failure to cleanup would be a problem - - this is part of the reason no STACKs were used, the other part of - the reason being I found them inappropriate). - - (iv) ensure that all the built-in crypto was encapsulated by one of - these "ENGINE"s and that this engine was automatically selected as - the default. - - (v) provide the minimum hooking possible in the existing crypto code - so that global functions (eg. RSA_public_encrypt) do not need any - extra parameter, yet will use whatever the current default ENGINE - for that RSA key is, and that the default can be set "per-key" - and globally (new keys will assume the global default, and keys - without their own default will be operated on using the global - default). NB: Try and make (v) conflict as little as possible with - (ii). :-) - - (vi) wrap the ENGINE code up in duct tape so you can't even see the - corners. Ie. expose no structures at all, just black-box pointers. - - (v) maintain internally a list of ENGINEs on which a calling - application can iterate, interrogate, etc. Allow a calling - application to hook in new ENGINEs, remove ENGINEs from the list, - and enforce uniqueness within the global list of each ENGINE's - "unique id". - - (vi) keep reference counts for everything - eg. this includes storing a - reference inside each RSA structure to the ENGINE that it uses. - This is freed when the RSA structure is destroyed, or has its - ENGINE explicitly changed. The net effect needs to be that at any - time, it is deterministic to know whether an ENGINE is in use or - can be safely removed (or unloaded in the case of the other type - of reference) without invalidating function pointers that may or - may not be used indavertently in the future. This was actually - one of the biggest problems to overcome in the existing OpenSSL - code - implementations had always been assumed to be ever-present, - so there was no trivial way to get round this. - - (vii) distinguish between structural references and functional - references. - -A *little* detail ------------------ - -While my mind is on it; I'll illustrate the bit in item (vii). This idea -turned out to be very handy - the ENGINEs themselves need to be operated -on and manipulated simply as objects without necessarily trying to -"enable" them for use. Eg. most host machines will not have the necessary -hardware or software to support all the engines one might compile into -OpenSSL, yet it needs to be possible to iterate across the ENGINEs, -querying their names, properties, etc - all happening in a thread-safe -manner that uses reference counts (if you imagine two threads iterating -through a list and one thread removing the ENGINE the other is currently -looking at - you can see the gotcha waiting to happen). For all of this, -*structural references* are used and operate much like the other reference -counts in OpenSSL. - -The other kind of reference count is for *functional* references - these -indicate a reference on which the caller can actually assume the -particular ENGINE to be initialised and usable to perform the operations -it implements. Any increment or decrement of the functional reference -count automatically invokes a corresponding change in the structural -reference count, as it is fairly obvious that a functional reference is a -restricted case of a structural reference. So struct_ref >= funct_ref at -all times. NB: functional references are usually obtained by a call to -ENGINE_init(), but can also be created implicitly by calls that require a -new functional reference to be created, eg. ENGINE_set_default(). Either -way the only time the underlying ENGINE's "init" function is really called -is when the (functional) reference count increases to 1, similarly the -underlying "finish" handler is only called as the count goes down to 0. -The effect of this, for example, is that if you set the default ENGINE for -RSA operations to be "cswift", then its functional reference count will -already be at least 1 so the CryptoSwift shared-library and the card will -stay loaded and initialised until such time as all RSA keys using the -cswift ENGINE are changed or destroyed and the default ENGINE for RSA -operations has been changed. This prevents repeated thrashing of init and -finish handling if the count keeps getting down as far as zero. - -Otherwise, the way the ENGINE code has been put together I think pretty -much reflects the above points. The reason for the ENGINE structure having -individual RSA_METHOD, DSA_METHOD, etc pointers is simply that it was the -easiest way to go about things for now, to hook it all into the raw -RSA,DSA,etc code, and I was trying to the keep the structure invisible -anyway so that the way this is internally managed could be easily changed -later on when we start to work out what's to be done about these other -abstractions. - -Down the line, if some EVP-based technique emerges for adequately -encapsulating algorithms and all their various bits and pieces, then I can -imagine that "ENGINE" would turn into a reference-counting database of -these EVP things, of which the default "openssl" ENGINE would be the -library's own object database of pre-built software implemented algorithms -(and such). It would also be cool to see the idea of "METHOD"s detached -from the algorithms themselves ... so RSA, DSA, ElGamal, etc can all -expose essentially the same METHOD (aka interface), which would include -any querying/flagging stuff to identify what the algorithm can/can't do, -its name, and other stuff like max/min block sizes, key sizes, etc. This -would result in ENGINE similarly detaching its internal database of -algorithm implementations from the function definitions that return -interfaces to them. I think ... - -As for DSOs etc. Well the DSO code is pretty handy (but could be made much -more so) for loading vendor's driver-libraries and talking to them in some -generic way, but right now there's still big problems associated with -actually putting OpenSSL code (ie. new ENGINEs, or anything else for that -matter) in dynamically loadable libraries. These problems won't go away in -a hurry so I don't think we should expect to have any kind of -shared-library extensions any time soon - but solving the problems is a -good thing to aim for, and would as a side-effect probably help make -OpenSSL more usable as a shared-library itself (looking at the things -needed to do this will show you why). - -One of the problems is that if you look at any of the ENGINE -implementations, eg. hw_cswift.c or hw_ncipher.c, you'll see how it needs -a variety of functionality and definitions from various areas of OpenSSL, -including crypto/bn/, crypto/err/, crypto/ itself (locking for example), -crypto/dso/, crypto/engine/, crypto/rsa, etc etc etc. So if similar code -were to be suctioned off into shared libraries, the shared libraries would -either have to duplicate all the definitions and code and avoid loader -conflicts, or OpenSSL would have to somehow expose all that functionality -to the shared-library. If this isn't a big enough problem, the issue of -binary compatibility will be - anyone writing Apache modules can tell you -that (Ralf? Ben? :-). However, I don't think OpenSSL would need to be -quite so forgiving as Apache should be, so OpenSSL could simply tell its -version to the DSO and leave the DSO with the problem of deciding whether -to proceed or bail out for fear of binary incompatibilities. - -Certainly one thing that would go a long way to addressing this is to -embark on a bit of an opaqueness mission. I've set the ENGINE code up with -this in mind - it's so draconian that even to declare your own ENGINE, you -have to get the engine code to create the underlying ENGINE structure, and -then feed in the new ENGINE's function/method pointers through various -"set" functions. The more of the code that takes on such a black-box -approach, the more of the code that will be (a) easy to expose to shared -libraries that need it, and (b) easy to expose to applications wanting to -use OpenSSL itself as a shared-library. From my own explorations in -OpenSSL, the biggest leviathan I've seen that is a problem in this respect -is the BIGNUM code. Trying to "expose" the bignum code through any kind of -organised "METHODs", let alone do all the necessary bignum operations -solely through functions rather than direct access to the structures and -macros, will be a massive pain in the "r"s. - -Anyway, I'm done for now - hope it was readable. Thoughts? - -Cheers, -Geoff - - ------------------------------------==*==----------------------------------- - -- cgit v1.2.3-55-g6feb