e12da35645
Fix spelling
2024-09-24 00:14:24 +10:00
cc16afb7f3
mrb_str_aset_m() should return replace instead of str
...
`string[]=(idx, replace)` should return `replace`.
## Actual (wrong)
```
string.[]=(idx, replace) → string
string.[]=(idx, len, replace) → string
```
## Expected
```
string.[]=(idx, replace) → replace
string.[]=(idx, len, replace) → replace
```
## Sidenote
As of the current mruby-compiler, `(string[idx] = 'X')` creates not only "CALL_NODE" but also "ASGN_NODE" and "OP_MOVE", overriding the wrong return value.
On the other hand, `string.[]=(idx, 'X')` creates only "CALL_NODE", exposing the wrong return value.
If my new mruby-compiler2, leveraging Prism, took the place of official compiler, `(string[idx] = 'X')` and `string.[]=(idx, 'X')` would be going to generate the same VM code without "OP_MOVE".
So I paranoidly added tests.
FYI: You can find how the new mruby-compiler2's AST and VM code look like in mruby/c's issue (mruby/c had the same bug): https://github.com/mrubyc/mrubyc/pull/210
2024-07-16 18:55:26 +09:00
a871e4908f
Minor cleanup in mrb_str_init
...
mrb_str_new was essentially manually inlined here, and I think it is a bit easier to follow if we just call the function instead.
2024-07-13 15:38:14 +02:00
c592e7b70d
string.c: small refactoring
2024-06-21 10:11:32 +09:00
7b9b0997e0
string.c: adjust local variable declarations
2024-05-31 22:49:25 +09:00
c258ead85b
Include headers for malloc() explicitly; fix #6277
...
We need to include stdlib.h and malloc.h to use malloc()/free() but
they aren't included in src/string.c with WIN32_LEAN_AND_MEAN. It
generates build time warnings.
We can solve this by including stdlib.h and malloc.h explicitly.
2024-05-29 09:45:36 +09:00
68e093729b
string.c: refactor bit operation (e.g. SWAR)
...
To generate a bit mask of `0x8080...` of the word size, we prepare
`MASK01` which is `0x0101...` and multiply masking word (e.g., `0x80).
2024-05-25 16:39:32 +09:00
bd7efbe75f
string.c: check if the index points character boundary; fix #6267
2024-05-25 16:39:31 +09:00
86b1c8ff87
string.c (utf8_strlen): add assertion utf8_len <= byte_len
2024-05-03 07:50:40 +09:00
fa213a2d89
string.c (search_nonascii): prevent buffer overrun; ref #6255
2024-04-29 16:35:08 +09:00
714ef4c4fd
string.c (mrb_utf8_strlen): handle invalid UTF-8 sequence; fix #6255
...
Previous SWAR version assumes valid UTF-8 to count number of code points
in the string, but we need to handle invalid sequence as well. We now
use `search_nonascii` to skip counting single byte characters for
performance. The new version is even faster than SWAR version (probably
because `search_nonascii` uses SSE2 on Intel compatible CPU (which I use).
2024-04-29 15:37:22 +09:00
aa28f48c3b
string.c (search_nonascii): simplify fallback implementation by SWAR
2024-04-02 19:50:13 +09:00
cbb31e6a7c
string.c (mrb_utf8_strlen): use SWAR technique for performance
2024-03-30 22:50:12 +09:00
8130bbc479
string.c (mrb_str_len_to_dbl): should return zero for invalid string
...
We should not use uninitialized value; fix #6183
2024-02-24 23:03:15 +09:00
3eeeb8fabf
string.c (memsearch_swar): skip a local variable
2024-02-15 14:01:49 +09:00
9c840a68e2
string.c (memsearch_swar): boundary check by the latest position
2024-02-13 22:36:03 +09:00
abcad36bae
string.c (memsearch_swar): fixed a buffer overflow
2024-02-09 19:09:01 +09:00
868aeaf48c
string.c (memsearch_swar): fixed a buffer overflow
2024-02-09 18:52:44 +09:00
cac698fe88
string.c (memsearch_swar): changed the function name
...
For your information, SWAR stands for SIMD within a register.
2024-01-31 21:37:39 +09:00
f8527f0670
string.c (mrb_memsearch_ss): update a comment regarding alignment issue
2024-01-31 21:36:05 +09:00
d8144aef78
string.c: remove a comment regarding unsigned char*
2024-01-31 21:34:54 +09:00
363f795432
string.c: remove unnecessary type casts
...
We don't need `unsigned` for string pointers.
2024-01-31 15:06:04 +09:00
e61a2881a7
string.c (mrb_memsearch_ss): fix potential buffer overflow; fix #6158
...
- stop using `bigint*`
- stop integer pointer dereferences
- use `memcpy` to integer variables
- add reminder search for shorter patterns
2024-01-31 07:48:24 +09:00
219cfd63e1
string.c (mrb_memsearch_ss): update integer prefixes (LLU -> ULL)
2024-01-31 07:42:52 +09:00
6ba65ea0f0
string.c (mrb_memsearch_ss): update comment
2024-01-31 01:54:37 +09:00
44a5882bc2
string.c (mrb_memsearch_ss): remove useless alignment adjustment
...
Ref #6158
2024-01-31 01:27:10 +09:00
be5448fd16
string.c (mrb_memsearch): refactor m==1 (use memchr) case
2024-01-31 01:17:12 +09:00
d8e1aed646
string.c (search_nonascii): need not to check n>3 on 32bit platforms
2024-01-30 12:35:56 +09:00
e859b430cc
fixup! string.c (search_nonascii): add faster search using SSE2
2024-01-27 11:59:29 +09:00
dee1ded7b5
string.c (mrb_memsearch_ss): support bigendians
2024-01-27 06:23:16 +09:00
06d9a54760
string.c (mrb_memsearch): remove simple search and quick search
...
Since mrb_memsearch_ss() is fast enough for most of the cases, we try to
simplify the code.
2024-01-27 06:22:55 +09:00
2943ca2685
string.c (mrb_memsearch_ss): faster integer-wise substring search
...
The function is based on @WojciechMula's code from the repository
https://github.com/WojciechMula/sse4-strstr.git Since it's licensed
under 2 clause BSD, we updated LEGAL file too.
2024-01-27 06:16:33 +09:00
08cd281200
string.c (search_nonascii): skip alignment adjustment for some CPUs
...
Some CPUs (e.g. x86) allow unaligned access to the memory.
2024-01-26 08:22:33 +09:00
68ab95ea57
string.c (mrb_memsearch): take char* instead of void*
...
Avoid useless `void*`.
2024-01-25 22:32:31 +09:00
cf0cd6785c
string.c: rename MRB_QS_SHORT_STRING_LENGTH
...
The new name is MRB_SEARCH_SHORT_STRING_LENGTH since it's no longer used
in the mrb_memsearch_qs(). FYI, 'qs' stands for 'quick search'.
2024-01-23 23:28:45 +09:00
fb2ebcde7c
string.c (search_nonascii): add faster search using SSE2
2024-01-23 23:28:45 +09:00
4108b85c62
string.c (search_nonascii): faster search using integer match
...
If you define `SIMPLE_SEARCH_NONASCII`, you can use old, naive
implementation of search_nonascii(). You may want to use the old one for
code size constraint for example.
2024-01-23 23:28:45 +09:00
772f13520c
Fixed buffer overflow in mrb_memsearch()
2024-01-20 09:44:47 +09:00
f13101124c
string.c: add cast to remove warnings
2024-01-19 18:24:34 +09:00
69cf074778
string.c (mrb_memsearch): move simple search from mrb_memsearch_qs()
2024-01-19 15:21:43 +09:00
57fd0edaa7
mruby.h: rename ASCII flag to SINGLE_BYTE
...
This flag means all the characters in the string can be represented by a
single byte, i.e., the string does not contain any multi-byte character.
Those characters are likely ASCII characters, but may be a part of broken
UTF-8 sequence, so the term 'ASCII' is not sufficient.
2024-01-18 22:50:39 +09:00
fb8bc1954b
string.c: improve performance of chars2bytes/bytes2chars; ref #6143
2024-01-11 18:23:50 +09:00
f646228dbe
string.c (str_index_str_by_char): simplify using str_index_str()
...
Instead of its own version of quick search, now we use str_index_str()
and adjust character position. This change makes searching 4 times
faster in some cases; ref #6143
2024-01-11 12:23:06 +09:00
e42f3b36f9
string.c (chars2bytes): simplify the condition to detect break
2024-01-11 11:49:49 +09:00
787439455c
string.c: remove the macro BYTES_ALIGN_CHECK which is no longer used
2024-01-10 17:31:04 +09:00
c2f148e15d
string.c (mrb_memsearch_qs): reduce the scope of local variables
2024-01-10 16:21:39 +09:00
31f2d93657
string.c (bytes2chars): skip calling mrb_utf8len() if possible
...
If (ch < 0x80) the length of the character (in bytes) should be 1, so we
don't have to call mrb_utf8len().
2024-01-10 15:36:07 +09:00
4c859e754d
string.c (bytes2chars): skip scanning if the string is ASCII only
2024-01-10 14:40:39 +09:00
df0a5e838d
string.c (chars2bytes): use early return
2024-01-10 12:41:28 +09:00
e39e4ea26d
string.c (chars2bytes): small refactoring
2024-01-10 11:09:47 +09:00
John Bampton