summaryrefslogtreecommitdiff
path: root/src/lib/libcrypto/bn/arch (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Rename old assembly bn_sqr_words() to bn_sqr_word_wise().jsing2025-09-073-6/+3
| | | | | | | bn_sqr_words() does not actually compute the square of the words, it only computes the square of each individual word - rename it to reflect reality. Discussed with tb@
* Constify bn_mul_words().jsing2025-09-011-2/+3
|
* Provide bn_mul_words() on amd64.jsing2025-09-012-2/+12
| | | | | This uses s2n-bignum's bignum_mul() and provides significant performance gains for a range of multiplication sizes.
* Reorder functions since they've been renamed.jsing2025-08-311-17/+17
|
* Rename bn_mul_words()/bn_mul_add_words().jsing2025-08-305-17/+17
| | | | | | | | | | | | | | Most bn_.*_words() functions operate on two word arrays, however bn_mul_words() and bn_mul_add_words() operate on one word array and multiply by a single word. Rename these to bn_mulw_words() and bn_mulw_add_words() to reflect this, following naming scheme that we use for primitives. This frees up bn_mul_words() to actually be used for multiplying two word arrays. Rename bn_mul_normal() to bn_mul_words(), which will then become one of the possible assembly integration points. ok tb@
* Rework bn_sqr() to use bn_sqr_words().jsing2025-08-302-13/+11
| | | | | | | | | | | | Rework some of the squaring code so that it calls bn_sqr_words() and use this as the integration point for assembly. Convert bn_sqr_normal() to bn_sqr_words(), which is then used on architectures that do not provide their own version. This means that we resume using the assembly version of bn_sqr_words() on i386, mips64 and powerpc, which can provide considerable performance gains. ok tb@
* Use faster versions of bignum_{mul,sqr}_{4_8,6_12,8_16}() if possible.jsing2025-08-141-10/+41
| | | | | | | | If ADX instructions are available, use the non-_alt version of s2n-bignum's bignum_{mul,sqr}_{4_8,6_12,8_16}(), which are faster than the _alt non-ADX versions. ok tb@
* Provide amd64 specific versions of bn_mul_comba6() and bn_sqr_comba6().jsing2025-08-142-2/+22
| | | | | | | These use s2n-bignum's bignum_mul_6_12_alt() and bignum_sqr_6_12_alt() functions. ok tb@
* Provide bn_mod_add_words() and bn_mod_sub_words() on amd64.jsing2025-08-142-2/+25
| | | | | | These use s2n-bignum's bignum_modadd() and bignum_modsub() routines. ok tb@
* Revise include to match the name that we use.jsing2025-08-1210-20/+20
|
* Replace SPDX-License-Identifier with actual license.jsing2025-08-1210-20/+130
|
* Add RCS tags to new files.jsing2025-08-1210-0/+20
|
* Bring in bignum_mod{add,sub}() from s2n-bignum.jsing2025-08-122-0/+185
| | | | These provide modular addition and subtraction.
* Bring in bignum_{mul,sqr}_{4_8,8_16}() from s2n-bignum.jsing2025-08-124-0/+877
| | | | | | | These provide fast multiplication and squaring of inputs with 4 words or 8 words, producing an 8 or 16 word result. These versions require the CPU to support ADX instructions, while the _alt versions that have previously been imported do not.
* Bring in bignum_{mul,sqr}_6_12{,_alt}() from s2n-bignum.jsing2025-08-124-0/+807
| | | | | | These provide fast multiplication and squaring of inputs with 6x words, producing a 12 word result. The non-_alt versions require the CPU to support ADX instructions, while the _alt versions do not.
* Add const to bignum_*() function calls.jsing2025-08-121-16/+16
| | | | | Now that s2n-bignum has marked various inputs as const, we can do the same. In most cases we were casting away const, which we no longer need to do.
* Add RCS tags.jsing2025-08-1111-0/+22
|
* Resync s2n-bignum primitives for amd64 with upstream.jsing2025-08-1111-115/+113
| | | | This amounts to whitespace changes and label renaming.
* Mark the inputs to bn_mul_comba{4,8}() as const.jsing2025-08-051-3/+3
| | | | | | | This makes it consistent with bn_sqr_comba{4,8}() and simplifies an upcoming change. ok tb@
* Provide an optimised bn_subw() for amd64.jsing2024-03-261-3/+22
| | | | bn_subw() will be used more widely in an upcoming change.
* Replace uses of endbr64 with _CET_ENDBR from cet.htb2024-02-2411-11/+11
| | | | | | | | | cet.h is needed for other platforms to emit the relevant .gnu.properties sections that are necessary for them to enable IBT. It also avoids issues with older toolchains on macOS that explode on encountering endbr64. based on a diff by kettenis ok beck kettenis
* Mark read/write registers as non-overlapping for bn_qwmulw_{addqw_,}addw()jsing2023-07-241-3/+3
| | | | | | This does not cause an issue currently, however if called differently to their current usage, it can lead to an input being overwritten and incorrect results being generated.
* Provide optimised bn_subw() for riscv64.jsing2023-07-091-1/+18
|
* Provide optimised bn_addw() for riscv64.jsing2023-07-091-1/+18
|
* Provide optimised bn_mulw() for riscv64.jsing2023-07-071-7/+11
| | | | | This provides a 1.5-2x performance gain for BN multiplication, with a similar improvement being seen for RSA operations.
* Rewrite and simplify bn_sqr()/bn_sqr_normal().jsing2023-06-241-3/+3
| | | | | | | | | Rework bn_sqr()/bn_sqr_normal() so that it is less convoluted and more readable. Instead of recomputing values that the caller has already computed, pass it as an argument. Avoid branching and remove duplication of variables. Consistently use a_len and r_len naming for lengths. ok tb@
* Provide optimised bn_subw() and bn_subw_subw() for arm.jsing2023-06-241-1/+50
|
* Provide optimised bn_clzw() for aarch64.jsing2023-06-211-1/+15
|
* Optimise bn_mul2_mulw_addtw() for aarch64.jsing2023-06-171-1/+28
| | | | | This provides significant performance gains for bn_sqr_comba4() and bn_sqr_comba8().
* Optimise quad word primitives on aarch64.jsing2023-06-121-1/+136
| | | | This provides a performance gain across most BN operations.
* Provide optimised bn_mulw_{addw,addw_addw,addtw}() for aarch64.jsing2023-05-281-1/+68
| | | | | This results in bn_mul_comba4() and bn_mul_comba8() requiring ~30% less instructions than they did previously.
* Provide optimised bn_addw_addw()/bn_subw_subw() for aarch64.jsing2023-05-281-1/+43
|
* Add endbr64 where needed by inspection. Passes regresson tests.deraadt2023-04-2511-0/+11
| | | | ok jsing, and kind of tb an earlier version
* Tweak indent and use named registers.jsing2023-04-171-13/+13
| | | | No functional change.
* Use separate lines instead of semicolons.bcook2023-02-251-4/+10
| | | | | | macOS aarch64 assembly dialect treats ; as comment instead of a newline ok tb@, jsing@
* Rework bn_add()/bn_sub() to operate on word arrays.jsing2023-02-221-7/+10
| | | | | | | | Rather than working on BIGNUMs, change bn_add()/bn_sub() to operate on word arrays that potentially differ in length. This matches the behaviour of s2n-bignum's bignum_add() and bignum_sub(). ok tb@
* Provide optimised versions of bn_addw() and bn_subw() for aarch64.jsing2023-02-171-1/+35
|
* Enable s2n-bignum word_clz() on amd64.jsing2023-02-162-2/+12
| | | | | | | | | The BN_num_bits_word() function is a hot path, being called more than 80 million times during a libcrypto regress run. The word_clz() implementation uses five instructions to do the same as the generic code that uses more than 60 instructions. Discussed with tb@
* Change include from _internal_s2n_bignum.h to s2n_bignum_internal.h.jsing2023-02-161-1/+1
|
* Include the ISC license from s2n-bignum's LICENSE file.jsing2023-02-161-1/+12
|
* Bring in word_clz.S from s2n-bignum for amd64.jsing2023-02-161-0/+48
|
* Rename bn_umul_hilo() to bn_mulw().jsing2023-02-166-42/+42
| | | | | | | | | This keeps the naming consistent with the other bignum primitives that have been recently introduced. Also, use 1/0 intead of h/l (e.g. a1 instead of ah), as this keeps consistency with other primitives and allows for naming that works with double word, triple word and quadruple word inputs/outputs. Discussed with tb@
* Remove bn_sqr_words() on amd64.jsing2023-02-042-11/+2
| | | | | | | s2n-bignum's bignum_sqr() is not the same as bn_sqr_words() (which only computes a partial result, unlike the former). This went unnoticed since bn_sqr() is called directly on amd64, hence bn_sqr_words() is currently unused.
* Fix output constraints for bn_umul_hilo().jsing2023-02-044-8/+8
| | | | | | | | When bn_umul_hilo() is implemented using an instruction pair, mark the first output with a constraint that prevents the output from overlapping with the inputs ("&"). Otherwise the first instruction can overwrite the inputs, which then results in the second instruction producing incorrect value.
* Refactor BN_uadd() and BN_usub().jsing2023-02-022-2/+22
| | | | | | | | | | | | | | | | | | Unlike bn_add_words()/bn_sub_words(), the s2n-bignum bignum_add() and bignum_sub() functions correctly handle inputs with differing word lengths. This means that they can be called directly, without needing to fix up any remaining words manually. Split BN_uadd() in two - the default bn_add() implementation calls bn_add_words(), before handling the carry for any remaining words. Likewise split BN_usub() in two - the default bn_sub() implementation calls bn_sub_words(), before handling the borrow for any remaining words. On amd64, provide an implementation of bn_add() that calls s2n-bignum's bignum_add() directly, similarly with an implementation of bn_sub() that calls s2n-bignum's bignum_sub() directly. ok tb@
* Provide inline assembly bn_umul_hilo() for alpha/powerpc64/riscv64.jsing2023-01-313-3/+67
| | | | | | These should work, but are currently untested and disabled. ok tb@
* Provide inline assembly versions of bn_umul_hilo() for aarch64/amd64/i386.jsing2023-01-313-3/+67
| | | | ok tb@
* Remove sparc related files from libcrypto.jsing2023-01-311-39/+0
| | | | | | | | The sparc platform got retired a while back, however some parts remained hiding in libcrypto. Mop these up (along with the bn_arch.h that I introduced). Spotted by and ok tb@
* Use s2n-bignum assembly implementations for libcrypto bignum on amd64.jsing2023-01-291-1/+79
| | | | | | | This switches the core bignum assembly implementations from x86_64-gcc.c to s2n-bignum for amd64. ok miod@ tb@
* Provide bn_div_rem_words() and make use of it.jsing2023-01-282-2/+52
| | | | | | | | | | | | | | | | Provide a function that divides a double word (h:l) by d, returning the quotient q and the remainder r, such that q * d + r is equal to the numerator. Call this from the three places that currently implement this themselves. This is implemented with some slight indirection, which allows for per architecture implementations, replacing the define/macro tangle, which messes with variables that are not passed to it. Also remove a duplicate of bn_div_words() for the BN_ULLONG && BN_DIV2W case - this is already handled. ok tb@
generated by cgit v1.2.3-55-g6feb (git 2.39.1) at 2025-12-24 16:57:20 +0000