1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2022-11-09 12:17:21 -05:00
Commit graph

23 commits

Author SHA1 Message Date
nobu
b65b392e96 tables.rb: add
* lib/unicode_normalize/tables.rb: commit not to download and
  convert Unicode data files every time.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-11 17:41:53 +00:00
nobu
9b559f194c normalize.rb: fix syntax error
* lib/unicode_normalize/normalize.rb (normalized): fix syntax
  error, `when` clase allows `*` but not `**`.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-09 10:01:37 +00:00
duerst
62b511b6aa lib/unicode_normalize/normalize.rb: Replaced if-else by case in self.normalized? in parallel to r48309.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48338 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-09 09:33:36 +00:00
nobu
d436c05163 unicode_norm_gen.tmpl: expand kompatible_table
* template/unicode_norm_gen.tmpl: expand kompatible_table so that
  recursive expansion is not needed at runtime.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48311 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-06 15:00:24 +00:00
nobu
b8788417f0 normalize.rb: trivial optimizations
* lib/unicode_normalize/normalize.rb (nfc_one, normalize): trivial
  optimizations.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-06 15:00:17 +00:00
nobu
64034372b7 normalize.rb: explicit separator
* lib/unicode_normalize/normalize.rb (canonical_ordering_one):
  use explicit separator, not to depend on $,.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-06 15:00:14 +00:00
duerst
2b7f0289f8 lib/unicode_normalize/normalize.rb: Comment clarification. [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48290 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-05 23:49:55 +00:00
duerst
4fda619836 lib/unicode_normalize/normalize.rb: added US_ASCII
as trivially supported encoding (is always normalized,
and may appear mixed in with UTF-8 or other Unicode
encodings).

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48134 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-25 11:09:08 +00:00
nobu
696141dab4 lib/unicode_normalize/tables.rb: remove auto generated file.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48074 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-21 13:48:05 +00:00
duerst
5c27164d59 lib/unicode_normalize/tables.rb: Committing to make version
update easier and more predictable, and reducing compilation
time.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48072 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-21 08:12:20 +00:00
duerst
7415796ca3 lib/unicode_normalize/normalize.rb: Added comment to point to
relevant portion of Unicode standard for Hangul (de)composition
identifiers and algorithm.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48071 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-21 06:56:58 +00:00
nobu
e64a3869bc unicode_normalize/normalize.rb: rename variable
* lib/unicode_normalize/normalize.rb (hangul_decomp_one): use more
  descriptive name.  leave [SLVT]BASE and [LVTNS]COUNT as they are
  vague names.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48055 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 11:50:00 +00:00
duerst
acaafe2101 lib/unicode_normalize.rb: revert r48046. The s in sIndex
is not hungarian notation. The variable name sIndex is
directly taken from the relevant part of the Unicode
Standard, where it is written SIndex and stands for
'syllable index'. See pp. 144/145 of
http://www.unicode.org/versions/Unicode7.0.0/ch03.pdf.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48052 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 10:06:11 +00:00
nobu
6948188f38 unicode_normalize/normalize.rb: remove prefix
* lib/unicode_normalize/normalize.rb (hangul_decomp_one): remove
  system hungarian prefix, nonsense in ruby.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48046 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 05:01:02 +00:00
nobu
7f652dc6cf unicode_normalize/normalize.rb: simplify
* lib/unicode_normalize/normalize.rb (NF_HASH_{D,C,K}): remove
  first element by Hash#shift.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48045 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 05:00:58 +00:00
nobu
3a2f81cf9a unicode_normalize/normalize.rb: remove unnecessary module names
* lib/unicode_normalize/normalize.rb (UnicodeNormalize): use self
  instead of module name and remove module name if unnecessary.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48044 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 05:00:46 +00:00
nobu
51af3be356 lib/unicode_normalize.rb: remove BOMs
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48028 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 15:29:58 +00:00
duerst
8c722a9a1e lib/unicode_normalize/normalize.rb: Added a missing file extension in require statement.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48022 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 09:35:45 +00:00
duerst
982f0de141 tool/unicode_norm_gen.rb, lib/unicode_normalize.rb:
File name change from lib/unicode_normalize/normalize_tables.rb
to lib/unicode_normalize/tables.rb.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48015 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 02:09:13 +00:00
svn
4bf30d2944 * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48009 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 00:48:55 +00:00
duerst
4c769ce021 lib/unicode_normalize/normalize.rb: Changed module name, adjusted copyright.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 00:48:52 +00:00
svn
d64dc54e0c * properties.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48007 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 00:38:53 +00:00
duerst
6017de0314 lib/unicode_normalize/normalize.rb: Importing from
https://github.com/duerst/eprun/blob/master/lib/normalize.rb.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48005 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 00:38:40 +00:00