aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorThijs Schreijer <thijs@thijsschreijer.nl>2024-05-23 20:46:18 +0200
committerThijs Schreijer <thijs@thijsschreijer.nl>2024-05-23 20:57:20 +0200
commit56db1511baeb0376a12915c69c1552b04010c26f (patch)
treed03aa6b4c33a6de39371e9be336c471bfd2cafc5
parent8f8d34f03428dbaa6cac229bbe36efc6d80d186d (diff)
downloadluasystem-56db1511baeb0376a12915c69c1552b04010c26f.tar.gz
luasystem-56db1511baeb0376a12915c69c1552b04010c26f.tar.bz2
luasystem-56db1511baeb0376a12915c69c1552b04010c26f.zip
cleanup and documentation
-rw-r--r--doc_topics/03-terminal.md124
-rw-r--r--examples/readline.lua38
-rw-r--r--examples/terminalsize.lua3
-rw-r--r--src/term.c19
-rw-r--r--system/init.lua7
5 files changed, 161 insertions, 30 deletions
diff --git a/doc_topics/03-terminal.md b/doc_topics/03-terminal.md
new file mode 100644
index 0000000..06a6b96
--- /dev/null
+++ b/doc_topics/03-terminal.md
@@ -0,0 +1,124 @@
1# 3. Terminal functionality
2
3Terminals are fundamentally different on Windows and Posix. So even though
4`luasystem` provides primitives to manipulate both the Windows and Posix terminals,
5the user will still have to write platform specific code.
6
7To mitigate this a little, all functions are available on all platforms. They just
8will be a no-op if invoked on another platform. This means that no platform specific
9branching is required (but still possible) in user code. The user must simply set
10up both platforms to make it work.
11
12## 3.1 Backup and Restore terminal settings
13
14Since there are a myriad of settings available;
15
16- `system.setconsoleflags` (Windows)
17- `system.setconsolecp` (Windows)
18- `system.setconsoleoutputcp` (Windows)
19- `system.setnonblock` (Posix)
20- `system.tcsetattr` (Posix)
21
22Some helper functions are available to backup and restore them all at once.
23See `termbackup`, `termrestore`, `autotermrestore` and `termwrap`.
24
25
26## 3.1 Terminal ANSI sequences
27
28Windows is catching up with this. In Windows 10 (since 2019), the Windows Terminal application (not to be
29mistaken for the `cmd` console application) supports ANSI sequences. However this
30might not be enabled by default.
31
32ANSI processing can be set up both on the input (key sequences, reading cursor position)
33as well as on the output (setting colors and cursor shapes).
34
35To enable it use `system.setconsoleflags` like this:
36
37 -- setup Windows console to handle ANSI processing on output
38 sys.setconsoleflags(io.stdout, sys.getconsoleflags(io.stdout) + sys.COF_VIRTUAL_TERMINAL_PROCESSING)
39 sys.setconsoleflags(io.stderr, sys.getconsoleflags(io.stderr) + sys.COF_VIRTUAL_TERMINAL_PROCESSING)
40
41 -- setup Windows console to handle ANSI processing on input
42 sys.setconsoleflags(io.stdin, sys.getconsoleflags(io.stdin) + sys.CIF_VIRTUAL_TERMINAL_INPUT)
43
44
45## 3.2 UTF-8 in/output and display width
46
47### 3.2.1 UTF-8 in/output
48
49Where (most) Posix systems use UTF-8 by default, Windows internally uses UTF-16. More
50recent versions of Lua also have UTF-8 support. So `luasystem` also focusses on UTF-8.
51
52On Windows UTF-8 output can be enabled by setting the output codepage like this:
53
54 -- setup Windows output codepage to UTF-8; 65001
55 sys.setconsoleoutputcp(65001)
56
57Terminal input is handled by the [`_getwchar()`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/getchar-getwchar) function on Windows which returns
58UTF-16 surrogate pairs. `luasystem` will automatically convert those to UTF-8.
59So when using `readkey` or `readansi` to read keyboard input no additional changes
60are required.
61
62### 3.2.2 UTF-8 display width
63
64Typical western characters and symbols are single width characters and will use only
65a single column when displayed on a terminal. However many characters from other
66languages/cultures or emojis require 2 columns for display.
67
68Typically the `wcwidth` function is used on Posix to check the number of columns
69required for display. However since Windows doesn't provide this functionality a
70custom implementation is included based on [the work by Markus Kuhn](http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c).
71
722 functions are provided, `system.utf8cwidth` for a single character, and `system.utf8swidth` for
73a string. When writing terminal applications the display width is relevant to
74positioning the cursor properly. For an example see the [`examples/readline.lua`](../examples/readline.lua.html) file.
75
76
77## 3.3 reading keyboard input
78
79### 3.3.1 Non-blocking
80
81There are 2 functions for keyboard input (actually 3, if taking `system._readkey` into
82account): `readkey` and `readansi`.
83
84`readkey` is a low level function and should preferably not be used, it returns
85a byte at a time, and hence can leave stray/invalid byte sequences in the buffer if
86only the start of a UTF-8 or ANSI sequence is consumed.
87
88The preferred way is to use `readansi` which will parse and return entire characters in
89single or multiple bytes, or a full ANSI sequence.
90
91On Windows the input is read using [`_getwchar()`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/getchar-getwchar) which bypasses the terminal and reads
92the input directly from the keyboard buffer. This means however that the character is
93also not being echoed to the terminal (independent of the echo settings used with
94`system.setconsoleflags`).
95
96On Posix the traditional file approach is used, which:
97
98- is blocking by default
99- echoes input to the terminal
100- requires enter to be pressed to pass the input (canonical mode)
101
102To use non-blocking input here's how to set it up:
103
104 -- setup Windows console to disable echo and line input (not required since _getwchar is used, just for consistency)
105 sys.setconsoleflags(io.stdin, sys.getconsoleflags(io.stdin) - sys.CIF_ECHO_INPUT - sys.CIF_LINE_INPUT)
106
107 -- setup Posix by disabling echo, canonical mode, and making non-blocking
108 local of_attr = sys.tcgetattr(io.stdin)
109 sys.tcsetattr(io.stdin, sys.TCSANOW, {
110 lflag = of_attr.lflag - sys.L_ICANON - sys.L_ECHO,
111 })
112 sys.setnonblock(io.stdin, true)
113
114
115Both functions require a timeout to be provided which allows for proper asynchronous
116code to be written. Since the underlying sleep method used is `system.sleep`, just patching
117that function with a coroutine based yielding one should be all that is needed to make
118the result work with asynchroneous coroutine schedulers.
119
120### 3.3.2 Blocking input
121
122When using traditional input method like `io.stdin:read()` (which is blocking) the echo
123and newline properties should be set on Windows similar to Posix.
124For an example see [`examples/password_input.lua`](../examples/password_input.lua.html).
diff --git a/examples/readline.lua b/examples/readline.lua
index f1e6258..286522c 100644
--- a/examples/readline.lua
+++ b/examples/readline.lua
@@ -1,3 +1,9 @@
1--- An example class for reading a line of input from the user in a non-blocking way.
2-- It uses ANSI escape sequences to move the cursor and handle input.
3-- It can be used to read a line of input from the user, with a prompt.
4-- It can handle double-width UTF-8 characters.
5-- It can be used asynchroneously if `system.sleep` is patched to yield to a coroutine scheduler.
6
1local sys = require("system") 7local sys = require("system")
2 8
3 9
@@ -134,7 +140,7 @@ readline.__index = readline
134--- Create a new readline object. 140--- Create a new readline object.
135-- @tparam table opts the options for the readline object 141-- @tparam table opts the options for the readline object
136-- @tparam[opt=""] string opts.prompt the prompt to display 142-- @tparam[opt=""] string opts.prompt the prompt to display
137-- @tparam[opt=80] number opts.max_length the maximum length of the input 143-- @tparam[opt=80] number opts.max_length the maximum length of the input (in characters, not bytes)
138-- @tparam[opt=""] string opts.value the default value 144-- @tparam[opt=""] string opts.value the default value
139-- @tparam[opt=`#value`] number opts.position of the cursor in the input 145-- @tparam[opt=`#value`] number opts.position of the cursor in the input
140-- @tparam[opt={"\10"/"\13"}] table opts.exit_keys an array of keys that will cause the readline to exit 146-- @tparam[opt={"\10"/"\13"}] table opts.exit_keys an array of keys that will cause the readline to exit
@@ -425,29 +431,25 @@ end
425 431
426 432
427 433
428-- return readline 434-- return readline -- normally we'd return here, but for the example we continue
435
429 436
430 437
431 438
439local backup = sys.termbackup()
432 440
433-- setup Windows console to handle ANSI processing 441-- setup Windows console to handle ANSI processing
434local of_in = sys.getconsoleflags(io.stdin)
435local cp_in = sys.getconsolecp()
436-- sys.setconsolecp(65001)
437sys.setconsolecp(850)
438local of_out = sys.getconsoleflags(io.stdout)
439local cp_out = sys.getconsoleoutputcp()
440sys.setconsoleoutputcp(65001)
441sys.setconsoleflags(io.stdout, sys.getconsoleflags(io.stdout) + sys.COF_VIRTUAL_TERMINAL_PROCESSING) 442sys.setconsoleflags(io.stdout, sys.getconsoleflags(io.stdout) + sys.COF_VIRTUAL_TERMINAL_PROCESSING)
442sys.setconsoleflags(io.stdin, sys.getconsoleflags(io.stdin) + sys.CIF_VIRTUAL_TERMINAL_INPUT) 443sys.setconsoleflags(io.stdin, sys.getconsoleflags(io.stdin) + sys.CIF_VIRTUAL_TERMINAL_INPUT)
444-- set output to UTF-8
445sys.setconsoleoutputcp(65001)
443 446
444-- setup Posix terminal to use non-blocking mode, and disable line-mode 447-- setup Posix terminal to disable canonical mode and echo
445local of_attr = sys.tcgetattr(io.stdin)
446local of_block = sys.getnonblock(io.stdin)
447sys.setnonblock(io.stdin, true)
448sys.tcsetattr(io.stdin, sys.TCSANOW, { 448sys.tcsetattr(io.stdin, sys.TCSANOW, {
449 lflag = of_attr.lflag - sys.L_ICANON - sys.L_ECHO, -- disable canonical mode and echo 449 lflag = sys.tcgetattr(io.stdin).lflag - sys.L_ICANON - sys.L_ECHO,
450}) 450})
451-- setup stdin to non-blocking mode
452sys.setnonblock(io.stdin, true)
451 453
452 454
453local rl = readline.new{ 455local rl = readline.new{
@@ -467,10 +469,4 @@ print("Exit-Key (bytes):", key:byte(1,-1))
467 469
468 470
469-- Clean up afterwards 471-- Clean up afterwards
470sys.setnonblock(io.stdin, false) 472sys.termrestore(backup)
471sys.setconsoleflags(io.stdout, of_out)
472sys.setconsoleflags(io.stdin, of_in)
473sys.tcsetattr(io.stdin, sys.TCSANOW, of_attr)
474sys.setnonblock(io.stdin, of_block)
475sys.setconsolecp(cp_in)
476sys.setconsoleoutputcp(cp_out)
diff --git a/examples/terminalsize.lua b/examples/terminalsize.lua
index 78d1910..ed66792 100644
--- a/examples/terminalsize.lua
+++ b/examples/terminalsize.lua
@@ -26,11 +26,12 @@ end
26 26
27local w, h 27local w, h
28print("Change the terminal window size, press any key to exit") 28print("Change the terminal window size, press any key to exit")
29while not sys.readkey(0.2) do 29while not sys.readansi(0.2) do -- use readansi to not leave stray bytes in the input buffer
30 local nw, nh = sys.termsize() 30 local nw, nh = sys.termsize()
31 if w ~= nw or h ~= nh then 31 if w ~= nw or h ~= nh then
32 w, h = nw, nh 32 w, h = nw, nh
33 local text = "Terminal size: " .. w .. "x" .. h .. " " 33 local text = "Terminal size: " .. w .. "x" .. h .. " "
34 io.write(text .. cursor_move_horiz(-#text)) 34 io.write(text .. cursor_move_horiz(-#text))
35 io.flush()
35 end 36 end
36end 37end
diff --git a/src/term.c b/src/term.c
index 99d3b2b..7020f09 100644
--- a/src/term.c
+++ b/src/term.c
@@ -337,7 +337,7 @@ To see flag status and constant names check `listconsoleflags`.
337Note: not all combinations of flags are allowed, as some are mutually exclusive or mutually required. 337Note: not all combinations of flags are allowed, as some are mutually exclusive or mutually required.
338See [setconsolemode documentation](https://learn.microsoft.com/en-us/windows/console/setconsolemode) 338See [setconsolemode documentation](https://learn.microsoft.com/en-us/windows/console/setconsolemode)
339@function setconsoleflags 339@function setconsoleflags
340@tparam file file the file-handle to set the flags on 340@tparam file file file handle to operate on, one of `io.stdin`, `io.stdout`, `io.stderr`
341@tparam bitflags bitflags the flags to set/unset 341@tparam bitflags bitflags the flags to set/unset
342@treturn[1] boolean `true` on success 342@treturn[1] boolean `true` on success
343@treturn[2] nil 343@treturn[2] nil
@@ -378,8 +378,17 @@ static int lst_setconsoleflags(lua_State *L)
378 378
379/*** 379/***
380Gets console flags (Windows). 380Gets console flags (Windows).
381The `CIF_` and `COF_` constants are available on the module table. Where `CIF` are the
382input flags (for use with `io.stdin`) and `COF` are the output flags (for use with
383`io.stdout`/`io.stderr`).
384
385_Note_: See [setconsolemode documentation](https://learn.microsoft.com/en-us/windows/console/setconsolemode)
386for more information on the flags.
387
388
389
381@function getconsoleflags 390@function getconsoleflags
382@tparam file file the file-handle to get the flags from. 391@tparam file file file handle to operate on, one of `io.stdin`, `io.stdout`, `io.stderr`
383@treturn[1] bitflags the current console flags. 392@treturn[1] bitflags the current console flags.
384@treturn[2] nil 393@treturn[2] nil
385@treturn[2] string error message 394@treturn[2] string error message
@@ -433,8 +442,8 @@ The terminal attributes is a table with the following fields:
433 442
434- `iflag` input flags 443- `iflag` input flags
435- `oflag` output flags 444- `oflag` output flags
436- `cflag` control flags
437- `lflag` local flags 445- `lflag` local flags
446- `cflag` control flags
438- `ispeed` input speed 447- `ispeed` input speed
439- `ospeed` output speed 448- `ospeed` output speed
440- `cc` control characters 449- `cc` control characters
@@ -528,9 +537,6 @@ flags for the `iflags`, `oflags`, and `lflags` bitmasks.
528To see flag status and constant names check `listtermflags`. For their meaning check 537To see flag status and constant names check `listtermflags`. For their meaning check
529[the manpage](https://www.man7.org/linux/man-pages/man3/termios.3.html). 538[the manpage](https://www.man7.org/linux/man-pages/man3/termios.3.html).
530 539
531_Note_: not all combinations of flags are allowed, as some are mutually exclusive or mutually required.
532See [setconsolemode documentation](https://learn.microsoft.com/en-us/windows/console/setconsolemode)
533
534_Note_: only `iflag`, `oflag`, and `lflag` are supported at the moment. The other fields are ignored. 540_Note_: only `iflag`, `oflag`, and `lflag` are supported at the moment. The other fields are ignored.
535@function tcsetattr 541@function tcsetattr
536@tparam file fd file handle to operate on, one of `io.stdin`, `io.stdout`, `io.stderr` 542@tparam file fd file handle to operate on, one of `io.stdin`, `io.stdout`, `io.stderr`
@@ -722,6 +728,7 @@ directly, but through the `system.readkey` or `system.readansi` functions. It
722will return the next byte from the input stream, or `nil` if no key was pressed. 728will return the next byte from the input stream, or `nil` if no key was pressed.
723 729
724On Posix, `io.stdin` must be set to non-blocking mode using `setnonblock` 730On Posix, `io.stdin` must be set to non-blocking mode using `setnonblock`
731and canonical mode must be turned off using `tcsetattr`,
725before calling this function. Otherwise it will block. No conversions are 732before calling this function. Otherwise it will block. No conversions are
726done on Posix, so the byte read is returned as-is. 733done on Posix, so the byte read is returned as-is.
727 734
diff --git a/system/init.lua b/system/init.lua
index b9a4f6f..8049167 100644
--- a/system/init.lua
+++ b/system/init.lua
@@ -7,7 +7,7 @@ local sys = require 'system.core'
7do 7do
8 local backup_mt = {} 8 local backup_mt = {}
9 9
10 --- Returns a backup of terminal setting for stdin/out/err. 10 --- Returns a backup of terminal settings for stdin/out/err.
11 -- Handles terminal/console flags, Windows codepage, and non-block flags on the streams. 11 -- Handles terminal/console flags, Windows codepage, and non-block flags on the streams.
12 -- Backs up terminal/console flags only if a stream is a tty. 12 -- Backs up terminal/console flags only if a stream is a tty.
13 -- @return table with backup of terminal settings 13 -- @return table with backup of terminal settings
@@ -227,8 +227,11 @@ do
227 -- This function uses `system.sleep` to wait until either a byte is available or the timeout is reached. 227 -- This function uses `system.sleep` to wait until either a byte is available or the timeout is reached.
228 -- The sleep period is exponentially backing off, starting at 0.0125 seconds, with a maximum of 0.2 seconds. 228 -- The sleep period is exponentially backing off, starting at 0.0125 seconds, with a maximum of 0.2 seconds.
229 -- It returns immediately if a byte is available or if `timeout` is less than or equal to `0`. 229 -- It returns immediately if a byte is available or if `timeout` is less than or equal to `0`.
230 --
231 -- Using `system.readansi` is preferred over this function. Since this function can leave stray/invalid
232 -- byte-sequences in the input buffer, while `system.readansi` reads full ANSI and UTF8 sequences.
230 -- @tparam number timeout the timeout in seconds. 233 -- @tparam number timeout the timeout in seconds.
231 -- @treturn[1] integer the key code of the key that was received 234 -- @treturn[1] byte the byte value that was read.
232 -- @treturn[2] nil if no key was read 235 -- @treturn[2] nil if no key was read
233 -- @treturn[2] string error message; `"timeout"` if the timeout was reached. 236 -- @treturn[2] string error message; `"timeout"` if the timeout was reached.
234 function sys.readkey(timeout) 237 function sys.readkey(timeout)