1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2022-11-09 12:17:21 -05:00
Commit graph

23 commits

Author SHA1 Message Date
akr
7a0bea4fd3 * include/ruby/encoding.h (rb_econv_t): add fields: in_buf_start,
in_data_start, in_data_end, in_buf_end and last_trans_index.
  (rb_econv_output): removed.
  (rb_econv_insert_output): declared.
  (rb_econv_encoding_to_insert_output): declared.

* enc/trans/newline.trans (rb_universal_newline): stateful_type
  changed.

* transcode.c (transcode_restartable0): initialize inchar_start,
  tc->recognized_len and next_table at beginning of the loop.
  (rb_econv_open_by_transcoder_entries): initialize new fields.
  (rb_econv_open): setup last_trans_index.
  (trans_sweep): last out_buf_start can be non-NULL now.
  (rb_econv_convert): check last out_buf_start and in_buf_start at
  first.
  (rb_econv_output_with_destination_encoding): removed.
  (econv_just_convert): removed.
  (rb_econv_output): removed.
  (econv_primitive_output): method removed.
  (rb_econv_encoding_to_insert_output): new function.
  (allocate_converted_string): new function.
  (rb_econv_insert_output): new function.
  (econv_primitive_insert_output): new method.
  (output_replacement_character): use rb_econv_insert_output.  unused
  arguments removed.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-16 05:32:42 +00:00
akr
c3d0206e04 add a test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18648 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 16:41:16 +00:00
akr
3a427c6199 * transcode.c (econv_just_convert): extracted from rb_econv_output.
(rb_econv_output): use econv_just_convert.
  (econv_primitive_output): new method.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18647 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 14:17:11 +00:00
akr
131e194cec * include/ruby/encoding.h (rb_econv_t): new field: last_error.
* transcode.c (rb_trans_conv): new argument: result_position_ptr.
  (rb_econv_convert): fill last_error.
  (econv_result_to_symbol): extracted from econv_primitive_convert.
  (econv_primitive_errinfo): new method.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 09:12:56 +00:00
akr
e4e2ef3975 add tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 02:32:47 +00:00
akr
8b04d998ae add a test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18636 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 00:27:03 +00:00
akr
b9d5d6a137 add tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18635 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 00:18:42 +00:00
akr
520cd6d186 * transcode.c (econv_init): don't create dummy encoding if
rb_econv_open is failed.
  (make_dummy_encoding): new function extracted from make_encoding.
  (make_encoding): removed.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18634 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-15 00:05:06 +00:00
akr
8eda80ff2c * transcode.c (econv_primitive_convert): set destination_buffer
encoding.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18630 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 16:06:33 +00:00
akr
c82aee31b4 * include/ruby/encoding.h (rb_econv_result_t): moved from
transcode_data.h.
  (rb_econv_elem_t): ditto.
  (rb_econv_t): ditto.  source_encoding and destination_encoding field
  is added.
  (rb_econv_open): declared.
  (rb_econv_convert): ditto.
  (rb_econv_close): ditto.

* transcode.c (rb_econv_open_by_transcoder_entries): initialize
  source_encoding and destination_encoding field as NULL.
  (rb_econv_open): make it external linkage.
  (rb_econv_close): ditto.
  (rb_econv_convert): ditto.  renamed from rb_econv_conv.
  (make_encoding): new function.
  (econv_init): use make_encoding and store rb_encoding* in
  rb_econv_t.
  (econv_source_encoding): new method
  Encoding::Converter#source_encoding.
  (econv_destination_encoding): new method
  Encoding::Converter#destination_encoding.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18625 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 14:28:10 +00:00
akr
a6daaf16b0 * transcode.c (econv_init): accept Encoding object as source_encoding
and destination_encoding.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18623 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 12:35:19 +00:00
akr
bd9bd52954 * transcode_data.h (transcode_invalid_byte_sequence): renamed from
transcode_invalid_input.
  (transcode_destination_buffer_full): renamed from transcode_obuf_full.
  (transcode_source_buffer_empty): renamed from transcode_ibuf_empty.
  (rb_econv_result_t): renamed from rb_trans_result_t.
  (rb_econv_elem_t): renamed from rb_trans_elem_t.
  (rb_econv_t): renamed from rb_trans_t.

* transcode.c (UNIVERSAL_NEWLINE_DECODER): renamed from
  UNIVERSAL_NEWLINE.
  (CRLF_NEWLINE_ENCODER): renamed from CRLF_NEWLINE.
  (CR_NEWLINE_ENCODER): renamed from CR_NEWLINE.
  (rb_econv_open): renamed from rb_trans_open.
  (rb_econv_close): renamed from rb_trans_close.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18618 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 11:31:27 +00:00
akr
7db5e184c2 add a test.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18616 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 10:38:41 +00:00
akr
a99e6c15b8 * transcode.c (rb_econv_conv): new function. it don't consume input
too much, even for multilevel conversion.
  (transcode_loop): use rb_econv_conv.
  (econv_primitive_convert): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18610 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 06:35:33 +00:00
akr
bcae240f19 * transcode_data.h (rb_trans_result_t): new enumeration constant:
transcode_output_followed_by_input.

* transcode.c (OUTPUT_FOLLOWED_BY_INPUT): new flag.
  (transcode_restartable0): suspend when output followed by input if
  OUTPUT_FOLLOWED_BY_INPUT is specified.
  (trans_sweep): check OUTPUT_FOLLOWED_BY_INPUT.
  (rb_trans_conv): support OUTPUT_FOLLOWED_BY_INPUT.
  (econv_primitive_convert): return :output_followed_by_input for
  transcode_output_followed_by_input.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18608 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 06:12:27 +00:00
akr
81be10bd85 test updated.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-14 00:40:25 +00:00
akr
fde26b7d7a * transcode.c (econv_primitive_convert): add output_byteoffset
argument.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18591 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-13 17:24:42 +00:00
akr
4aa251999d * transcode.c (econv_init): make flags argument optional.
(econv_primitive_convert): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18562 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-13 06:08:56 +00:00
akr
a14d5eb09b * enc/trans/newline.trans (rb_crlf_newline): new transcoder.
(rb_cr_newline): new transcoder.

* transcode.c (trans_open_i): one more exra room for input newline
  converter.
  (rb_trans_open): crlf newline and cr newline implemented.
  (Init_transcode): Encoding::Converter::CRLF_NEWLINE and
  Encoding::Converter::LF_NEWLINE defined.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18557 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-13 05:48:57 +00:00
akr
74a2a7bdbf * enc/trans/newline.trans: new file.
* transcode_data.h (rb_trans_t): add last_tc field.

* transcode.c (UNIVERSAL_NEWLINE): defined.
  (CRLF_NEWLINE): defined.
  (CR_NEWLINE): defined.
  (rb_trans_open_by_transcoder_entries): initialize last_tc.
  (trans_open_i): allocate one more room for newline converter.
  (rb_trans_open): universal newline implemented.
  (more_output_buffer): take max_output argument instead ts.
  (output_replacement_character): take tc argument instead of ts.
  (transcode_loop): use last_tc field.
  (econv_init): add flags argument for rb_trans_open.
  (Init_transcode): Encoding::Converter::UNIVERSAL_NEWLINE defined.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18556 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-13 05:30:42 +00:00
akr
5501d84ceb * transcode.c (econv_primitive_convert): add output_size argument.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18548 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-12 23:03:46 +00:00
akr
9d4bb3b969 * transcode.c (rb_trans_conv): report last transcode_obuf_full.
(econv_max_output): new method Encoding::Converter#max_output.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18547 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-12 22:43:17 +00:00
akr
edcd02cb5a * transcode.c (rb_cEncodingConverter): new class Encoding::Converter.
(econv_free): new function.
  (econv_s_allocate): ditto.
  (econv_init): ditto.
  (check_econv): ditto.
  (econv_primitive_convert): new method.
  (Init_transcode): define Encoding::Converter.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18544 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2008-08-12 14:46:18 +00:00